This disclosure generally relates to systems and methods for determining the location (i.e., position and orientation) of an item of interest relative to a coordinate system of a target object and then identifying that item of interest. The item of interest may be a part of the target object (hereinafter “part of interest”). The target object may be an airplane.
Situations where location and part identification data is desirable are common in applications involving inspection, maintenance, and repair of airplanes in environments such as flight line operations, airplane-on-ground events and customer walks (inspections). It is common in these situations for on-site non-expert personnel to communicate with an off-site expert analyst regarding a problem or concern. It is common practice for support personnel to take photographs of damaged locations. Based on these photographs, an on-site or off-site expert analyst may attempt to define where the damage is located by visually comparing the image of the damaged location with available technical data such as drawings, 3-D models, manual illustrations and similar documentation. This gives a subjective indication of the location reference of the damaged location relative to a target object. Based on this location, analysis is conducted with a high probability of location error.
The communication of information by an on-site technician may include the transmission of digital photographs of the area of interest to the off-site expert analyst using a smartphone, digital camera, tablet, laptop, notebook personal computer (PC), or other hand-held imaging device. Typically, the digital image files contains metadata such as date and time information and camera setting information (such as the camera model and make, and information that varies with each image such as orientation (rotation), aperture, shutter speed, and focal length). This metadata is typically stored in fields having a standard format, such as the EXIF format. “EXIF” is an acronym for “exchangeable image file”, which is a standard that specifies the formats for images, sound and ancillary tags used by digital cameras (including smartphones), scanners and other systems handling image and sound files recorded by digital cameras.
It is often the case that a hand-held imaging device will have an integrated global positioning system (GPS) receiver, in addition to a camera and other sensors. These devices can add geographical location data into the digital image metadata fields, which procedure is commonly referred to as “geotagging”. The integration of this type of data in digital image metadata storage fields can be very useful in locating where a photograph was taken. However, without additional context about the location (i.e., position and orientation) of the target object relative to the GPS frame of reference, this geotagging data is not sufficient to determine the location of the imaging device relative to the target object. Since a geotagged image does not contain enough data to fully describe the location of the imaging device relative to the target object, the process of identifying a part of interest in the image can be difficult.
It would be advantageous to provide a method capable of identifying parts of target objects based on geotagged photographs of areas of interest using relatively inexpensive hand-held imaging devices.